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(57) Abstract 

A method and circuit for achieving minimum latency data transfer between two mesochronous (same frequency, different phase) 
clock domains is disclosed. This circuit supports arbitrary phase relationships between two clock domains and is tolerant of temperature and 
voltage shifts after initialization while maintaining the same output data latency. In one embodiment, this circuit is used on a bus-system 
to re-time data from rcceivc-domain. clocks to transmlt-<lomain clocks. In such a system the phase relationships between these two clocks 
is set by the device bus location and thus is not precisely known. By supporting arbitrary phase resynchronization. this disclosure allows 
for theoretically infinite bus-length and thus no limitation on device count, as well as arbitrary placement of devices along the bus. This 
ultimately allows support of multiple latency-domains for very long buses. 
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METKIOD AND APPARATUS FOR FADL-SAFE 
RESYMCHRONIZATIIOM WITH MDNIR/iUM LATENCY 



This application claims priority to the provisional patent application entitled 
"Method and Apparatus for Fail-Safe Resynchronlzation with P^inimum Latency", 
Serial Number 60/061 ,505, filed October 10, 1997. 



BACKGROUND OF THE INVENTION 



The present invention relates to circuits for synchronizing data transfer 
between clock domains of the same frequency but different phase, in particular for 
synchronous memory systems. 

IViesochronous systems consist of two or more different clocks running at the 

15 same frequency with an arbitrary phase relationship. In such systems 

synchronization failures-can occur when passing data from one clock domain into 
another. While this problem is similar to that of asynchronous systems, it is easier to 
solve in that both clocks are mnning at the same frequency. It is possible to achieve 
reliable lower latency data transfer than is possible in asynchronous systems because 

20 of this constraint. 

Computer systems typically use a bus to transmit data between separate 
components. In order to transmit data at the highest rates with the lowest latency, 
synchronous transmission is frequently used. In one embodiment (see Fig. 1), a 
transmit clock is sent along with the data signal down a bus until it reaches the 

25 receiving device where it can be used to clock the data. This is done so that the 

delay the signal sees when traveling down the bus wire is matched by the delay of the 
clock traveling down a matched bus wire. If the same technique is used to send clock 
and data in both directions, then the transmit and receive data (and clocks) can have 
phase relationships that depend on the position of the device along the bus. When 

30 crossing from the receive clock domain to the transmit clock domain inside the pari, it 
becomes necessary to re-time the data, or at least to re-time a control pulse to the 
transmit clock domain. Prior art implementations placed restrictions on the length of 



i wo 99/19806 PCT/US98/21448 . 

-2 - 

such buses. In bussed systems such as Fig. 1 , the length of a bus determines the 
phase relationship between receive and transmit clocks. 

It is convenient to view any phase difference between two mesochronous 
clocks as a fraction of the clock cycle time. With two clocks of cycle-time TCYCLE 
5 and with phase relative to the source defined as tTxClk for the transmit clock and 
tRCIk for the receive clock (see Fig. 5) then tTR Is the relative phase between the 
falling edges of the clocks as a fraction of the clock cycle time and can be 
represented as: 

hp 

'CVCtE 

With this definition, the phase position of two clocks with exactly the same 

1 0 relationship would be tTR = 0, and two clocks who are inverted from each other would 
be tTR = 0.5, etc. U.S. Patent No. 5,432,823 to Gasbarro uses a fixed latency 
protocol to provide output data at a consistent latency despite clock skews. Because 
of limitations in circuit performance, the maximum skew, and thus bus length in one 
embodiment of Gasbarro (see Fig. 2) has a tTR limit. If a device was placed beyond 

15 this limit, its output data latency could be uncertain, as the internal synchronization 
circuits would be near their failure point. Prior art, using a fixed-latency protocol does 
not provide control for the transition from one latency to another. To eliminate this 
occurrence, limitations are sometimes placed on the tTR range of operation, and thus 
potentially on the maximum bus length and device count. 

20 Another system is described in copending application Serial No. 08/897,658 of 

Portmann et. al., filed July 21, 1997, and commonly assigned with this application 
(see Fig, 3). Portmann describes chains of flip-flops which can gradually be used to 
re-time data from one domain to another. Such re-tlming systems are actually a 
subset of the arbitrary-phase problem; the skew between clocks in Portmann can be 

25 easily recreated by adding a matching delay chain on-chip. Such systems also suffer 
from large latency from input to output data, as each flip-flop in the chain can add 
another cycle of output latency. They also suffer from the same uncertainty at the 
latency domain switching-point as Gasban-o, and so are unsuitable for use in systems 
with arbitrary phase-relationships between clocks, such as would be encountered in 

30 long-buses. 

The present invention allows for unlimited tTR. By supporting a mixed-latency 
protocol it allows for tTR greater than 1 .0 without uncertainty concerns by creating 
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^ firm latency domains. Unlike prior art. It maintains the output latency chosen during 
initialization despite any Vdd or temperature variations that may later occur. In 
addition, this invention includes circuits that allow operation at higher frequencies than 
prior art. 

5 

SUIVIMARY OF THE INVENTION 

The present invention provides a synchronization circuit having two latching 
circuits which latch data from a receive clock domain on both the rising and falling 

10 edges of a transmit clock. The selection of which latch data to use is based on a 

phase measurement of the phase difference between the receive and transmit clocks, 
which provides a select signal to a multiplexer connected to the two latching circuits. 
The phase difference is chosen to provide data which is as close as possible to the 
middle of an overiap region of valid data for the two latches. Once this value has 

15 been chosen, it is frozen in a latch after initialization. Since the data could have one, 
two or more clock cycle latencies from its origin, freezing the clock selection also 
freezes the latency period for operation of the circuit. Thus, although the 
synchronization circuit only knows the relative phase difference, not the latency, it can 
establish a stable operating configuration. 

20 The terms "receive" and "transmif clock domains refer to reception and 

transmission by the synchronization circuit, and does not necessarily correspond to 
received and transmitted data by a device, such as a memory device. The invention 
is particulariy useful in a synchronous memory system in which multiple DRAIVi are 
placed along a synchronous bus, at varying latencies from a clock origination point. 

25 The re-clocked data at each individual DRAM could be in a first latency period or a 
second latency period. The overall system may be equalized by adding latency to 
devices having less of a latency period to thus provide that all the data will be placed 
on the bus at the same time. 

In a preferred embodiment, the receive clock domain data is provided through 

30 a flip-flop clocked by a receive clock. The two latching circuits are preferably flip-flops 
utilizing a precharged sense amp in order to minimize shifting of the clock due to 
set-up, hold and clock-to-data-out variations. 

Preferably, the phase measurement circuit has a phase comparator coupled to 
receive the receive clock and also a clock which is In quadrature with the transmit 
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clock, rather than the transmit clock itself, in order to place the ultimate skip signal at 
the point of maximum overlap of valid data. The output of the phase comparator Is 
provided to an integrator In order to remove any jitter and narrow any uncertainty in 
the decision point. Preferably, an external transmit clock is used, rather than one that 
has been synchronized using a DLL or PLL. in order to eliminate another possible 
source of jitter. For a further understanding of the nature and advantages of the 
invention, reference should be made to the following description taken in conjunction 
with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 
Fig. 1 Is a diagram of a system containing clocks sent with data. 
Fig. 2 is a diagram of a synchronization circuit of Gasbarro et. al. 
Fig. 3 is a diagram of a synchronization circuit of Portmann et. al. 
Fig. 4 is a diagram of mixed-latency domains. 
Fig. 5 is a diagram of clock relationships. 

Fig. 6 is a block diagram of a preferred embodiment of a resynchronization 
system. 

Fig. 7 is a diagram of input windows; RET-FF/FET-FF. 

Fig. 8 Is a diagram of comparator characteristics. 

Fig. 9 is a diagram of ladder ranges. 

Fig. 10 is a diagram of synchronizer circuit. 

Fig. 1 1 is a diagram of synchronizer timing at tTR=0.0. 

Fig. 12 is a diagram of synchronizer timing attTR=0.5. 

Fig. 13 is a diagram of synchronizer timing at 

Fig. 14 is a diagram of shifting ladders due to synchronizer circuit variations. 

Fig. 15 Is a diagram of the narrowing ladder due to circuit deadband. 

Fig. 16 is a diagram of source of ladder shift and shrinkage (ideal elements). 

Fig. 17 is a diagram of source of ladder shift and shrinkage (typical elements). 

Fig, 18 is a diagram of source of ladder shift and shrinkage (slow elements). 

Fig. 1 9 is a diagram of synchronizer partial embodiment. 

Fig. 20 is a diagram of an alternate synchronizer. 

Fig. 21 is a diagram of four-phase synchronizer. 

Fig. 22 Is a diagram of four-phase ladder tTr. 

Fig. 23 is a diagram of a blended ladder synchronizer. 
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data arrives in the other flip-flop's setup & hold region. The output of FET-FF 307 is 
then re-timed through latch 308 (see Fig. 10) so that both inputs 216 and 215 to 
multiplexer 310 have the same timing relationship. In this case the outputs are set to 
be rising-edge based; in an alternate embodiment, a negative-enable latch could be 
5 placed after RET-FF 309 instead and the output from the mux would be falling-edge 
based. The two data paths that create the inputs to mux 310 are named ladders, as 
the re-timing function they perfomi is akin to a "fish-ladder" used to aid fish in 
swimming-upstream. RET-FF 309 makes up Ladderl while FET-FF 307 and latch 
308 make up LadderO. 

10 

Overview of Skip Circuit 401 

The function of the Skip circuit 401 is to generate an output signal which 
represents the phase skew between two input clocks. It must do this with both 
precision and accuracy in its determination. A block diagram of the Skip Circuit is 

1 5 shown as Fig. 25. It generates its output via a phase-comparator 405 followed by an 
integrator 404 and then a latch 403. The phase-comparator compares the input 
phases by sampling one clock with the other (TxClk is sampled with RCIk in one 
embodiment). If the clocks skew is 0 < tTR < 0.5 (see Fig. 8) then the phase- 
comparator generates an output of 0. If the skew is 0.5 < tTR < 1 .0 then the phase- 

20 comparator generates an output of 1 . To the phase-comparator there is no difference 
between tTR of 0 and tTR of 1 , or between tTR of 0.25 and 1 .25, i.e. its behavior is 
cyclic with phase offset. The phase-comparator is designed with very high gain so its 
transitions are as close to tTR=0.5 and tTR=1 as possible. To reduce or eliminate the 
effects of jitter on either clock, the outputs of the comparator are fed into integrator 

25 404. The integrator can be of any conventional form, but contains an input LatchSkip 
219 which shuts off integration and ensures full-rail resolution if the Integrated value is 
analog. The integrator serves two purposes. First, it takes the mean of any gaussian 
jitter (noise) distribution on the input clocks. Secondly, it further narrows any 
uncertainty in the decision at tTR = 0.5 and 1 . Lastly, upon assertion of LatchSkip, 

30 the skip latch 403 is closed. This becomes Important in the application of the Skip 
Circuit to multi-latency operation, and will be discussed next. 
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Overview of Synchronizer & Skip Circuit Opera tion Together. Lona-Bus Operation 
The synchronization problem becomes one of proper selection between the 
LadderO and Ladderl via mux 310. This control of mux 310 is accomplished with the 
Skip Circuit 401 . In order to understand the requirements for the Skip Circuit, it is 

5 better to first understand the ladder behavior in more detail when used in an arbitrary 
phase system. Recall that tTR is the phase relationship between TxClk and RCIk. In 
bused systems, such as those in Fig. 1, where clocks travel in the same direction as 
data, it is often required that the clock traveling with the data have its edge located in 
the center of the valid data window. This can be accomplished through a variety of 

1 0 techniques usually involving a DLL or PLL. This results In the creation of a clock in 
quadrature (90° out of phase) with TxClk. We will refer to this clock in quadrature 
with TxClk as TCIk. This clock is needed In order to clock the final output stages 
which time the output data, with TxClk lagging TCIk by 90'. 

In Fig. 9 the working range of the ladders is represented with tTR as the X-axis. 

1 5 In order to understand this figure, It is critical to realize that the X-axis is NOT time; It 
is tTR, the normalized delay difference between TxCIk and RCIk. With this 
underetanding, it can be seen that the working range 232 of LadderO is close to one 
cycle time, in fact it is equal to TCYCLE - (setup + hold time of the flip-flop 307). It 
can also be noted that the placement of range 232 begins at (-0.25 + hold) and ends 

20 at (0.75 - setup). The reason that the starting and ending are at -0.25 and 0.75, and 
not O and 1 is that TCIk Is used for the ladders. TCIk is in quadrature with TxClk 
which is used to specify tTR. 

By observing the first valid operating range of LadderO (232) and of Ladderl 
(231) in Fig. 9 it can be seen that they overlap. The center of this overlap occurs at 

25 precisely tTR = 0.5. TTiis, then, is the ideal position for the skip-circuit to change from 
selection of LadderO to selection of ladderl . This location will provide maximum 
margin from either failure point in either ladder. Next, observe what happens to a 
ladder as It is pushed beyond Its valid operating range. By observing LadderO ranges 
232 and 233 one can see that just before tTR = 0.75, LadderO stops working due to 

30 the input data passing beyond Its valid data window, but It begins functioning again 
shortly beyond tTR = 0.75. In operating range 233 LadderO's output has moved an 
entire cycle. That is. output from LadderO in range 233 will have one additional cycle 
of latency relative to its operation in range 232, and Is said to be in latency domain 2 
(see also Fig. 4). Both ladders, and Indeed any synchronization circuitry will exhibit 



.f 
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this behavior if the clock skews are pushed beyond their limit. The control of these 
different latency domains is a key innovation in this invention. 

While LadderO operates with latency of 1 and 2 in ranges 232 and 233, so 
does Ladderl within ranges 231 and 234, Recall that skip 230 Is used to seamlessly 

5 choose between LadderO and Ladderl at tTR of 0.5, Skip is also used to transition 
from Ladderl back to LadderO at tTR of 1 .0. While the Skip transition at tTR of 0.5 
was In no way visible at the output (i.e., the signals 236. 237, and 238 of Fig. 4 are 
identical), the outputs 236-238 are quite different from output 239. Although 238 and 
239 are shown at the same tTR of 1 .0, output 238 has skip=1 and when output 239 

10 has skip set back to 0, an additional cycle of latency is added to the output data. 

Since the resynchronization circuitry may be used in a system consisting of 
bused devices as in Fig. 1 , it is critical that the devices not change the latency of their 
output data if they happen to be placed at exactly a latency boundary. The skip latch 
403 (Fig. 25) is used to ensure that after a period of initialization, the value of skip is 

15 latched and can no longer change. This guarantees that the output latency will not 

change unless the circuit is reset by re-assertion of LatchSkip 219. After de-assertion 
of LatchSkip the output data latency can be guaranteed to not change. Fail-safe 
operation requires no synchronization failure in the synchronizer 400 as well as stable 
output latency. 

20 This requires, however, that the skip circuit transitions be placed with accuracy 

and precision, and that the synchronization circuitry have ladders with accurate 
placement, little deadband and with minimal movement across any voltage and 
temperature shifts. Any inaccuracy in the skip position or limits of valid ladder 
operation will limit the potential operating frequency, operating temperature range, or 

25 operating voltage if the Skip Circuit chooses a ladder which is pushed beyond Its valid 
operating range. This invention achieves those requirements to provide 
synchronization at highest operating frequency, over temperature and voltage 
variations, with the minimum latency possible for fail-safe operation. 

30 Overview of Generation of LatchSkip 

The use of the synchronizer and skip circuit in a system requires a control 
signal to freeze the output of the Skip Circuit 401 , called LatchSkip 219 (see Fig. 25). 
The assertion of this signal causes the skip value to be sampled and held inside latch 
403 of the Skip Circuit 401 . If the synchronizer is to be used in systems which span 



wo 99/19806 




PCT/US98/21448 



more than one clock cycle this function is nomnaliy accomplished during initialization, 
before the output of the synchronization circuit can be used. This avoids the 
possibility of outputs with different latencies. 

There are multiple possible ways of generating LatchSkip 219 to achieve this 

5 function. It can simply be asserted through a device reset. In this way when reset is 
de-asserted the skip output will be latched and the output latency will be fixed unless 
the device is reset. There are several other ways LatchSkip 219 can be generated. 
There are frequently initialization packets that are required to setup a device before 
use. In this case LatchSkip can be generated by a strobe used to receive one of 

1 0 these setup packets. By ensuring that a setup packet asserts LatchSkip 21 9, the skip 
output can be guaranteed to be stable before the synchronization circuit is actually 
used to re-time output data. Altemately, register or mode bits inside a device can 
also control the strobing of LatchSkip 219. Combining register control with other 
techniques allows for the flexibility of later skip evaluation and latching without 

1 5 requiring a device reset. 

Overview of Multiole Operating Frequencies 

In some applications it may be desirable for the system to be able to run at 

multiple different operating frequencies. This might be done in order to accommodate 
20 different power modes, where lower power can be obtained at lower frequencies by 

disabling some high-power circuitry. In such an application, it may be desirable to 

maintain multiple values of Skip, one for each operating frequency. In this case 

multiple skip latches and multiple LatchSkip signals can be used as shown in Fig. 26. 

In this system initialization would include the evaluation of the phase comparator and 
25 integrator at each operating frequency followed by latching into separate skip latches. 

The appropriate value of skip for the current operating frequency can then be 

selected by simply using the corresponding latch output. 

Detail of Svnchronization Circuit 400 
30 The Timing Synchronizer passes data between sequential circuits of two 

artDitrary-phased mesochronous clock domains. The synchronizer ensures signals 
are re-timed fail-safe or without metastability and timing glitches, and with minimum 
latency from circuits of one clock domain to circuits of another clock domain. In 
addition, when used to time output data transmission on a bus system such as one 
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r shown in Fig. 1 . the synchronizer times the output data so it arrives within a particular 
cycle at the destination. 

A preferred embodiment of the timing synchronizer is shown in Fig. 10. This 
circuit consists of Rclk flip-flop 306, Tclk flip-flops 307, 309, Tclk latch 308, and 

5 iViultiplexer 31 0. The synchronizer may be partitioned into two signal paths or ladders 
(LadderO. Ladderl), each with its own latency characteristics. Data propagates 
through LadderO via nodes 217. 603. 214. 216; Data propagates through Ladderl via 
nodes 217. 603. 215. Fig. 11 shows typical outputs at nodes 215 and 216 for 
tTR=0.0. (Note that TxClk lags TCIk by 90° in phase.) At this tTR. the output of 

10 LadderO is 1 clock cycle later than Ladderl . This result is typical of tTR= -0.25 to 

0.25. For the tTR=0.25 to 0.75. the outputs of LadderO and Ladderl are identical and 
indicate the overlap region of the synchronizer, as shown in Fig. 12. Between 
t-rR^O.75 and 1 .25 the synchronizer output (see Fig. 13) has moved to the next cycle 
akin to the relative ladder output positions in Fig. 1 1 . The behavior is thus cyclic with 

15 TCYCLE. 

In synchronizer 400 circuit operation, flip-flop 306 samples input Data In 217 
every RCIk cycle, and holds output 603 valid for one clock cycle (see Fig. 11). At the 
beginning of each clock cycle, the value at node 603 may transition depending on 
Datain 217. During this transition period, data cannot be reliably sampled by edge 

20 triggered flip-flops. Because TCIk flip-flop 307 and 309 sample data on the falling and 
rising edges of TCIk respectively, at least one of these two flip-flops is guaranteed to 
sample data from node 603 when it is not changing. This is required for fail-safe data 
resynchronization. For most ranges of tTR neither of the Tclk sampling edges are 
near node 603's data transition, and both Ladders will capture the correct value. 

25 However, over varying values of tTR, output from LadderO at node 21 6 may be zero 
or one cycle later than the output from Ladderl at node 21 5. 

Each ladder operates over a particular tTR range of operation without 
encountering problems with metastability. The range of valid tTR with which each 
ladder operates is detemnined by the range where a particular TCIk sampling edge 

30 212 falls within Valid Data window 603. For example, if the tTR value is 0.25 (in 
reference to Fig. 12, RCIk 21 1, Datain 217, and Valid Data 603 should be shifted 
earlier by 90°), the sampling rising edge of TCIk aligns with the tail portion of valid 
data 603. For cases where tTR is greater than 0.25, RCIk 211, Datain 21 7, and Valid 
Data 603 shifts later In time, until the sampling rising edge of TCIk aligns with the 
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beginning portion of valid data 603 at tTR =1 .25. The center of each ladder's 
operating range occurs when the center 613 of the Valid Data window 603 aligns with 
the ladder's sampling TCIk edge. The center of the overlap between the two ladders 
occurs when Valid Data Center 613 is equidistant from the two sampling TCIk edges 
as in Fig. 12. 

Region (a) of Fig. 14 shows the operating ranges of the two ladders in the 
Synchronizer circuits versus tTR phase values. (In the context of a device on a 
bus/channel such as that shown in Fig. 1, a particular tTR value may be interpreted 
as an indicator of the position on the bus.) As discussed in the previous paragraph. 
LadderO ideally operates for tTR range of -0.25 to 0.75 (see Fig. 14(a)/622). 
Similarly, Ladderl ideally operates for tTR range of 0.25 to 1.25 (see Fig. 14(a)/622). 
The overlap window between ladders 621 and 622 in indicates the region where both 
Ladderl and LadderO outputs are the same. LadderO 623 is similar to LadderO 621 . 
However, its output is shifted 1 Cycle later in time. The ends of each ladder shown in 
region (a) of Fig. 14 indicate tTR operation ranges beyond which the particular ladder 
is unreliable. 

Proper operation should avoid the ends of the ladders. Therefore, even with 
ideal components, LadderO 621 may be used for the entire range except at tTR=- 
0.25, and tTR=0.75; Ladderl 622 may be used for the entire range except at 
tTR=0.25, and tTR=1.25. In the overlap region, either LadderO or Ladderl may be 
used. For optimum fail-safe operation, the multiplexer 310 (Fig. 10) should select 
another ladder before the end of the current ladder is reached. Therefore, the optimal 
mux select 0 to 1 transition should be at tTR=0.5. and the optimal mux select 1 to 0 
transition should be at tTR=1 .0. This placement of mux select values over tTR leaves 
equal amounts of margin for timing errors of non-ideal ladders. These errors will be 
discussed in the next section. An important point to note is that the input clocks to the 
synchronizer are periodic, and thus synchronizer characteristics repeat in tTR space. 

Practical implementation of the synchronizer yields ladder circuits which do not 
fully correspond to region (a) of Fig. 14. Each ladder In region (a) of Fig. 14 spans the 
theoretical full clock cycle of tTR range, resulting in a half cycle of ladder overlap. 
However, in practice, the synchronizer has less overlap margin due to circuit 
imperfections. These imperfections take the fomi of shifting ladder alignment as 
shown in region (b) of Fig. 14, and narrowing the ladders as shown in Fig. 15. Thus, 
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> the ladders each span less than 1 clock cycle, the overlap region spans less than half 
a clock cycle, and the ladder positions may be shifted from the Ideal case. 

Analysis of the clock domain crossing indicates the sources of these errors. 
An examination of the signal flow between flip-flops 306 and 307 in Fig. 10 best 
5 illustrates some of the timing errors. In the ideal case, all flip-flops have zero setup- 
or-hold requirements, and no clock-to-data-out delay. In our example, RCIk flip-flop 
306 will have no clk-to-data-out delay as shown by waveform 702 (see Fig. 16). 
Waveform 704, 705 shows zero setup and hold for TCIk flip-flop 307. The ideal 
circuit elements in Fig. 16 will result in a one clock cycle long ladder span with 
10 symmetric overlap region between LadderO and Ladderl at both ends of the span. 
Actual flip-flops do not perfomi as well. Fig. 17 and Fig. 18 show the 
waveforms of flip-flops 306 and 307 (shown in Fig. 10) for a typical and slow 
perfonnance conditions respectively. The varying circuit perfomriance is due to 
varying integrated circuit transistor processes, power supply voltage and operating 
15 temperature of practical systems. Ladder position shifts are caused by clock-to-data- 
out delay variation of flip-flop 306 and setup/hold variations of flip-flop 307. These 
variations are equivalent to shifting Rclk and Tclk (thus effective tTR) in the ideal case 
shown in Fig. 16. Any mismatched clock buffering of Tclk and Rclk within the 
Integrated circuit contributes additional ladder shifts. 
20 Besides ladder shifts, another source of error Is narrowing of the ladder's valid 

operating region. This narrowing is caused by the data transition time 717 of circuit 
306, and setup 714/724 and hold 715/725 of flip-flop 307. Within this period, capture 
of signal 603 is unreliable and may lead to wrong data. Clock jitter also adds to 
ladder narrowing by injecting phase shifts to the clocks. Clock jitter adds to the 
25 uncertainty region caused by set-up, hold and clock-to-data-out 71 8/728 variations. 

Similar analysis applies to the clock domain boundary between flip-flops 306 and 309. 

The circuit implementation of the synchronizer addresses the aforementioned 
Ladder shifts and shrinkage for maximum timing margin over varying process, 
voltage, and temperature conditions. A partial circuit schematic of one 
30 implementation is shown in Fig. 19. This circuit addresses the sources of Ladder 
shifts and shrinkage with delay compensation and faster circuit perfonnance. To 
minimize ladder shifting, a precharged sense amp is used for the second stage flip- 
flops. The clocked sense-amp latching delay varies approximately with the clock-to- 
data-out delay of the previous stage latch. So in slower transistor performance 
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^ conditions, the clocked sense-amp will delay its sampling point to compensate for the 
added clock-to-data-out delay at nodes 806 and Ladder17. The use of differential 
signals 806. 807 instead of single-ended signal 603 increases the sense-amp gain, 
and lowers the setup and hold of flip-flops 307, 309. 
5 The Synchronizer circuit shown in Fig. 10 uses the falling edge of RCIk 

831/830 and the rising and falling edges of TCIk 808/809 to generate overlapping 
LadderO and Ladderl. This configuration generates rising-edge triggered outputs. In 
an alternate embodiment for a falling-edge driven sequential logic system, we could 
reverse the polarity of the sampling edge of all sequential elements, and achieve 

10 proper operation. Alternatively, the synchronizer can use the rising and falling edges 
of RCIk with one sampling edge of Tclk, an embodiment of which is shown in Fig. 20. 
The basic premise used to achieve clock domain crossing is to use circuits clocked 
with different phases of one clock domain to pass data into another domain. The end 
results are overlapping ladders similar to those shown in Fig. 14. 

1 5 Another embodiment of the timing synchronizer is shown in Fig. 21 , and 

consists of four overlapping ladders 961/971 , 962/972. 963/973 and 964/974. The 
resultant ladder ranges are shown versus tTR value in Fig. 22. We should note that 
the ideal overlap between each ladder is 3/4 TCYCLE. This allows substantially more 
margin for errors in ladder shifts, narrowing, and placements of Mux 979 select 

20 controls. This embodiment divides the synchronization problem into four separate 
ladders. If this concept is taken further, an arbitrary number of ladders (of two or 
greater) may be used with the ideal overlap margin being: 

Toverlap- Tcyclem^ia^dder^ 
num{ladders) 

The above equation shows diminishing increase of overlap margin with an increased 
number of ladders. Also, an increasing number of ladders requires a larger number 

25 of mux select controls. 

A different embodiment of the synchronizer is shown in Fig. 23. This 
synchronizer realigns the data with the use of a "blended" clock IntClk 1 005. The 
blended clock has a phase which is halfway between RCIk and TCIk as shown in Fig. 
24. The synchronizer includes three sequentially coupled latches 1003. 1006, 1009 

30 that are clocked by RCIk 1002. IntClk 1005 and TCIk 1008. respectively. 

Node 1004 represents RCIk valid data, which moves later in phase relative to 
TCIk as tTR increases. This synchronizer circuit preserves the output delay of Data 
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Out/1010 relative to TCIk over the entire tTR range of 0 to 1 . At tTR=0, the input 
clocks Rclk and TCIk are aligned, and the synchronizer has a three clock cycle delay 
through the circuit. As tTR increases, and RCIk/1002 and Valid Data/1004 lags TCIk. 
the Latency of the synchronizer circuit decreases to compensate for the increased 
5 delay of RClk relative to TCIk. Therefore. Data Out 1 01 0 is captured by the same 
TCIk edge even as the RClk valid data 1004 is delayed by up to one clock cycle. 

The inputs required for this synchronizer are RClk 1002, IntClk 1005» TCIk 
1008. and RCIk-domain Data In 1001 . An extemal multiplexer select generator is not 
required. In a preferred embodiment. IntClk 1 005 is generated by a phase 
1 0 interpolator or mixer circuit 1 007. 



Detail of Skip Circuit 401 

The Skip Circuit 401 (see Fig. 25) is made up of a phase comparator 405. an 
integrator 404. and a latch 403. The purpose of the Skip Circuit is to generate a very 

15 accurate measurement of tTR with minimal-width uncertainty regions. It also provides 
the capability to latch the output value after initialization, eliminating the possibility of 
multiple operating latencies if tTR is near a latency-domain boundary. 

A preferred embodiment of the phase-comparator in Fig. 27 is used in order to 
provide an accurate measurement with minimum uncertainty. It contains high-gain 

20 narrow-uncertainty-window data receivers of the same type that are used to sample 
bus data. In the preferred embodiment these are used in conjunction with a DLL or 
PLL which has aligned RClk so any receiver setup+hold time has been removed. 
Using a data receiver to sample the extemal signal TxClk has two advantages over 
sampling an internal clock. First, it gives the most accurate measurement of bus 

25 position as internal clocks can be skewed by clock recovery circuits for a number of 
reasons. Second, it provides the lowest-jitter source of transmit-domain phase 
information available. Any clock recovered with a DLL or PLL (e.g. TCIk) will 
necessarily have higher jitter than at the input clock. 

In the ideal case, a simple FET (falling-edge-triggered) data receiver could be 

30 used to generate the phase-comparator output. However, since clocks traveling at 
high-speed can undergo duty-cycle distortion, a single-data receiver used in this 
fashion could incon^ectly determine the value of tTR if it was sampling a duty-cycle 
distorted clock while being clocked by a duty-cycle corrected clock (see Fig. 28). In 
the case where RClk locks to the falling-edge of its corresponding bus clock, a TxClk 
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^ duty-cycle 248 less than 50% would cause a FET phase-comparator to falsely put the 
tTR = 0.5 transition at a tTR greater than 0.5. Similarly, if TxClk duty-cycle was 
greater than 50% 249, the comparator would erroneously put the tTR=0.5 transition at 
a tTR lower than 0.5. The use of an additional data-receiver 408 in Fig. 27 which is 

5 clocked on the rising edge of RCIk together with an inverter 41 2 avoids this problem. 
Near tTR of 0.5 where a single falling edge triggered receiver would be providing false 
information due the time shifted rising edge of TxClk. the rising edge of RCIk will be 
near the falling edge of TxClk. By choosing whichever data receiver is closest to 
sampling the falling-edge of TxClk, using multiplexer 410 and flip-flop 41 1 , the phase 

10 comparator can tolerate a non-50% duty-cycle on TxClk. This can be more easily 
understood by obsen/ing the different characteristics of waveforms 248. 249 and 251 
at the time of both the RCIk rising edge 253 and the RCIk falling edge 252. The 
preferred embodiment of Fig. 27 thus uses flip-flop 41 1 to make a rough comparison 
of TCIk and Rclk and generate a control signal, SelEven 247, that is used by 

1 5 multiplexer 41 0 to choose FET receiver 409 for -0.25 < tTR < 0.25 and RET receiver 
408 for 0.25 < tTR < 0.75. The exact tTR positioning of SelEven is not critical, as in 
most cases outputs 245 and 246 are identical. The prefen-ed embodiment thus can 
compare the phase of potentially duty-cycle distorted clocks to duty-cycle corrected 
clocks and maintain a high degree of accuracy in phase-comparison by only 

20 measuring to the falling-edge. In an alternate embodiment, the inverter could be 
moved and the multiplexer control switched to compare to the rising edge of TxClk 
with the same effect. 

An alternate embodiment of phase-comparator 405 uses a Quadrature Phase 
Detector to compare RCIk and TCIk, the Quadrature Phase Detector having output 

25 which Indicates when the clocks are in quadrature. This eliminates the need for the 
use of a clock which Is in quadrature to TCIk. and allows for the combined system to 
only use two clocks. 

The Skip Circuit integrator 404 consists of a standard integrator, which serves 
to narrow the uncertainty window created by a jitter distribution (such as a gaussian 

30 distribution). Any clock will have a certain amount of random noise which shows up 
as a distribution of phase. Using an integrator on the output of the phase-comparator 
ensures that signal 243's transition will be closest to the mean-point of the jitter 
distributions of both TxClk 220 and RCIk 21 1 . while at any single-point in time the raw 
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phase-comparator output 242 could be the result of comparison to an extreme of the 
clock jitter distributions. 

An integrator 404 preferred embodiment shown in Fig. 29 is an analog 
integrator with steered up and down current sources 416 and 435 into capacitors 417 
5 and 436 which create a differential voltage on output nodes 243 and 270. The 

steering of these currents is controlled by switches driven by the comparator output 
242. In a preferred embodiment, the integrator is allowed to run for at least 40 cycles 
after the clocks settle, in order to ensure an adequate gaussian distribution of the 
clock inputs. After this time, the LatchSkip 219 signal is asserted and de-asserted to 
1 0 both the integrator and the skip latch 403 turning off integration. If the integrator is 
analog, this signal also asserts a high-gain sense-amp 415, LatchSkip 219 also 
closes the skip latch 403 so the Skip Circuit output will not change unless LatchSkip 
219 is re-asserted. 

15 Detail of Combined Operation of Svnchronizer 400 and S kip Circuit 401 

The Synchronizer 400 and Skip Circuit 401 make up a synchronization system. 
Both elements are required in the preferred embodiment in order to properly 
synchronize data between two mesochronous clock domains. This invention allows 
for 1) high operating frequency and then 2) low latency while maintaining fail-safe 

20 operation. 

In order to achieve fail-safe operation, the Synchronizer must not fail to both 
properly pass and re-time the incoming data. The most significant failure mechanism 
is if the improper ladder for the current operating conditions (tTR, Voltage, 
temperature) is used. This can come about due to error sources which this invention 
25 minimizes. The multiple possible error sources result in an equation which limits the 
cycle-time of the system. Possible sources of error include: 

1, Error in the absolute placement of the ladder range 

This can result in the Skip Circuit choosing the ladder which is closer to 
its failure point. It can also result in less than optimal ladder overlap 
30 regions. 

2. Shifting of the ladders due to chances in operating environment (e.g. 
Vdd. temp) 

Ladder shift during operation must be minimized so that the functional 
ladder range does not pass the current operating tTR. 
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3. Ladder deadband 

Ladder deadband caused by flip-flop setup & hold uncertainty regions 
directly subtracts from the timing budget for data resynclironization. 
Clock jitter can also sen/e to shift the ladder ranges. 

4. Skip accuracy 

Error In the absolute placement of the skip transition can result in 
choosing the ladder which is closer to its failure point, similar to #1 . 

5. Skip uncertainty 

A large uncertainty-window in skip means that it is possible that skip will 
choose the ladder closer to its failure point. 



The preferred embodiment of both circuits minimizes each of the possible error 
sources to proyide the maximum fail-safe operating frequency. The sources of error 
listed above lead to an equation which must be maintained in order to haye fail-safe 
operation. This equation represents the minimum cycle-time limit of the system. 



Cycle ^ 



' T T ^ 

'Setup* 'Hold 



*Error^ag^*ErrorsMp*Movement^^g, 

^ ) . , 

The preferred embodiment places the Skip transition at tTR=1 . As previously 

mentioned, the placement of this transition sets the boundary of the first latency 

domain. In another embodiment. RCIk or TxClk/TCIk may be shifted earlier or later by 

a constant time or phase delay so the first latency domain boundary occurs at a tTR 

other than 1 . As both the Synchronizer 400 and the Skip Circuit 401 use RCIk as the 

starting reference both circuits will shift together. All latency domains remain 1 cycle 

wide, and so all boundary ends will move as well. 

An alternate embodiment of this nature that requires no delay elements 
receives the R-domain data 217 of Fig. 10 with a rising-edge-triggered flip flop instead 
of falling-edge-triggered flip-flop 306 and the value of skip is Inverted from that in the 
normal embodiment. By making these two changes the circuit moves the first latency 
domain range to -0.5 < tTR < 0.5 and the second range to 0.5 < tTR < 1.5. This may 
be done if a coarse adjustment of the domain start and end points is desired. 

Thus the combination of the Synchronizer and Skip circuit provide high 
frequency fail-safe synchronization for mesochronous systems such as that of Fig. 1 . 
Arbitrary phase delays are possible while fail-safe latency Is minimized and output 
latency is kept constant despite changes in the circuit's operating point. 
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Multiple Operating Frequencies 

To accommodate switching between multiple operating frequencies without 
requiring re-evaluation of skip an alternate embodiment of the Skip circuit 430 can be 
used as shown In Fig. 26. In this embodiment the single Skip Latch 403 of Fig. 25 
has been replaced by multiple skip latches 431 -433. In addition, the single LatchSkip 
signal 219 of Fig. 25 has been replaced by LatchSkip_F1 to LatchSkip_Fn 260-265. 
There is one latch and one LatchSkip signal for each desired operating frequency. 

The operation of this embodiment is as follows. Upon initialization, each of the 
desired operating frequencies is selected and initialized. During this initialization, 
after stabilization of the clocks at the new frequency the appropriate LatchSkip_Fx 
signal is asserted using one of the techniques described previously. After all 
frequencies have been initialized the correct value for Skip at each desired frequency 
is stored in the respective skip latches 431-433. The device can then simply select 
the appropriate Skip value depending on its current operating frequency without 
having to wait for the Skip Circuit to reevaluate. A multiplexing circuit can connect a 
selected skip latch to skip input 213 of Fig. 10. 
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1 . A synchronization circuit for synclironizing data between receive and transmit 
mesochronous clocks, comprising: 

a receive clocl< domain circuit for providing the data clocked by the receive 

clock; 

a first latching circuit, coupled to an output of the receive clock domain circuit, 
for latching the data on a first edge of the transmit clock; wherein the receive clock 
and transmit clock are mesochronous; 

a second latching circuit, in parallel with the first latching circuit, for latching the 
data on a second edge of the transmit clock; 

a multiplexing circuit, having inputs coupled to outputs of the first and second 

latching circuits; and 

a phase measurement circuit, configured to measure a phase difference 
between the receive and transmit clocks and to provide a select signal to the 
multiplexer in accordance with the phase difference. 

2. The synchronization circuit of claim 1 , wherein the phase measurement circuit 
determines a phase difference between the receive clock and a clock in quadrature 
with the transmit clock; the phase measurement circuit including a phase comparator 
coupled to receive the receive clock and the clock in quadrature with the transmit 
clock, and an integrator coupled to an output of the phase comparator; and 

the synchronization circuit including a latch coupled to an output of the phase 
measurement circuit for latching an output signal after an initialization period. 

3. The synchronization circuit of claim 1 , wherein the phase measurement circuit 
determines a phase difference between the receive clock and a clock in quadrature 
with the transmit clock. 

4. The synchronization circuit of claim 1 wherein the phase measurement circuit 
comprises: 

a phase comparator coupled to receive the receive clock and the clock in 
quadrature with the transmit clock; and 

an integrator coupled to an output of the phase comparator. 
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1 5. The synchronization circuit of claim 1 further comprising a latch coupled to an 

2 output of the phase measurement circuit for latching an output signal after an 

3 initialization period. 

1 6. The synchronization circuit of claim 1 wherein the latching circuits are flip-flops. 

1 7. The synchronization circuit of claim 1 further comprising a third latching circuit, 

2 connected between an output of the first latching circuit and the multiplexer, for re- 

3 timing the output of the first latching circuit to the second edge of the transmit clock 

4 signal. 

1 8. The synchronization circuit of claim 1 further comprising a select signal latch 

2 connected to a select input of the multiplexer. 

1 9. The synchronization circuit of claim 8, further comprising a plurality of select 

2 signal latches, for storing different select signals corresponding to different clock 

3 frequencies. 

1 10. The synchronization circuit of claim 1 wherein the first and second latching 

2 circuits each include a precharged sense amplifier. 

1 11. A synchronous memory bus system comprising: 

2 a data bus having a first end and a second end; 

3 a clock source generating a clock signal; 

4 a clock line carrying the clock signal, the clock line having 

5 a first clock line segment extending from the first end of the data bus to 

6 a turnaround near the second end of the data bus, and 

7 a second clock line segment extending from the turnaround to the first 

8 end of the data bus; 

9 a first device coupled to the data bus for transmitting data to a second device, 

10 including synchronization circuitry that receives a first transmit clock from one of the 

1 1 first and second clock line segments, a receive clock from the other one of the first 

12 and second clock line segments, and a second transmit clock in quadrature with the 
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13 first transmit clock, wherein the receive clock and first transmit clock are 

1 4 mesochronous, the synchronization circuitry comprising: 

1 5 a receive clock domain circuit for providing the data clocked by the receive 

1 6 clock; 

17 a first latching circuit, coupled to an output of the receive clock domain circuit. 

1 8 for latching the data on a first edge of the second transmit clock; 

-j 9 a second latching circuit, in parallel with the first latching circuit, for latching the 

20 data on a second edge of the second transmit clock; 

21 a multiplexing circuit, having inputs coupled to outputs of the first and second 

22 latching circuits; and 

23 a phase measurement circuit, configured to measure a phase difference 

24 between the receive clock and first transmit clock and to provide a select signal to the 

25 multiplexer in accordance with the phase difference. 

1 12. A method for synchronizing data between receive and transmit mesochronous 

2 clocks, comprising the steps of: 

3 providing data clocked by a receive clock from a receive clock domain; 

4 latching the data in a first latching circuit on a first edge of a transmit clock; 

5 latching the data in a second latching circuit on a second edge of the transmit 

6 clock; wherein the receive clock and transmit clock are mesochronous; 

7 measuring a phase difference between the receive and transmit clocks and 

8 providing a skip signal in accordance with the phase difference; and 

9 selecting between the first and second latching circuit based on the skip signal. 

1 13. The method of claim 1 2 wherein the step of measuring a phase difference 

2 comprises determining a phase difference between the receive clock and a clock in 

3 quadrature with the transmit clock. 

1 14. The method of claim 12 wherein the step of measuring a phase difference 

2 comprises the steps of: 

3 comparing the receive clock and a clock in quadrature with the transmit clock; 

4 and 

5 integrating a result of the comparing step. 
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^ 1 1 5. The method of claim 1 2 further comprising the step of latching the skip signal 

2 after an initialization or calibration sequence. 

3 1 6. The method of claim 1 2, further comprising latching an output of the first 

4 latching circuit with on the second edge of the transmit clock signal so as to re-time 

5 the output of the first latching circuit to the second edge of the transmit clock signal. 

1 17. A synchronization circuit for synchronizing data between receive and transmit 

2 mesochronous clocks, comprising: 

3 a first receive clock domain circuit for providing the data clocked by a first edge 

4 of the receive clock; 

5 a second receive clock domain circuit for providing the data clocked by a 

6 second edge of the receive clock; 

7 two latching circuits, coupled to an output of the received clock domain circuits, 

8 for latching the data on an edge of the transmit clock; wherein the receive clock and 

9 transmit clock are mesochronous; 

10 a multiplexing circuit, having inputs coupled to outputs of the two latching 

1 1 circuits; and 

12 a phase measurement circuit, configured to measure a phase difference 

13 between the received and transmit clocks and to provide a selection signal to the 

14 multiplexer in accordance with the phase differences, 

1 18. The synchronization circuit of claim 17 wherein the phase measurement circuit 

2 determines a phase difference between the receive clock and a clock in quadrature 

3 with the transmit clock. 

1 19. The synchronization circuit of claim 1 7 wherein the phase measurement circuit 

2 comprises: 

3 a phase comparator coupled to receive the receive clock and the clock in 

4 quadrature with the transmit clock; and 

5 an integrator coupled to an output of the phase comparator. 
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1 20. The synchronization circuit of claim 17 further comprising a latch coupled to an 

2 output of the phase measurement circuit for latching an output signal after an 

3 initialization period. 



1 



21 . A synchronization circuit for synchronizing data between receive and transmit 

2 mesochronous clocks, comprising: 

3 a receive clock domain circuit for providing the data clocked by the receive 

4 clock; 

5 an intermediate clock domain circuit latching the out'^^^^Jt of the receive clock 

6 domain circuit on an edge of an intermediate clock, wherein the intermediate clock is 

7 a phase-interpolated wavefomi of the receive and transmit clocks and the receive 

8 clock and transmit clock are mesochronous; and 

9 a transmit clock domain circuit, coupled to an output of the intemnediate clock 
10 domain circuit, for latching the data on an edge of the transmit clock. 

1 22. The circuit of claim 21 further comprising: 

2 a phase interpolator or mixer for generating the intermediate clock from the 

3 receive and transmit clocks. 



* > 



wo 99/19806 PCT/US98/21448 

1/25 



FIG 1 



Clk( 

r 

300 



201 200 

\ ClockFromMaster= CFM 



ClockToMaster = CTM 



Data Bus 202-^ 



Chip N 



Slave 



301 




303 



Clocks at Chip N 
TxClk = CTM 

RCIk phase-locked to CFM 



Chip 0 



302 



Master 




Clocks at Chip 0 

304 TxClk phase = RCIk 
(CTM phase = CFM) 



SUBSTITUTE SHEET (RULE 26) 



wo 99/119806 



PCT/US98/21448 



2/25 




o 



SUBSmUTESHEET (RULE 26) 



wo 99/19806 



PCT/US98/21448 



3/25 



FIG 3 



r 



-L Q D 



It 



Q D 
E 



Q D 
E 



O 0 
E 



Q 0 
E 



SUBSTITUTE SHEET (RULE 26) 



wo 99/19806 PCT/US98/21448 

4/25 

FIG 4 
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FIG 8 
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FIG 10 
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